Linux AIO example codes. March 30, 2005

ttcp.aio.c:
 
  Data sink/source program used to measure network throughput.
       
  libaio will need to be available on your system, new RHEL/Fedora
  distros contain it by default, or you can download and build it
  yourself:

    http://fr2.rpmfind.net/linux/rpm2html/search.php?query=libaio

  Next step build ttcp.aio.c, you will need to let gcc know where
  the sdp_sock.h include file is located. This file contains the
  definition of AF_INET_SDP and some socket options:

    gcc -I../../../linux-kernel/infiniband/ulp/sdp ttcp.aio.c \
        -o ttcp.aio.x -laio

  The program runs in two modes, a server which listens for incoming
  connections and acts as a data sink, and a client which connects to
  a given address/port and acts as a data source. A decent help exists
  for available parameters, but here are some reasonable defaults:

    server:

      ./ttcp.aio.x -r -l 65536 -a 20

    client:

      ./ttcp.aio.x -t -l 65536 -n 100000 -a 20 192.168.0.100

  Available parameters:

    Usage: ttcp.aio.x -t [-options] host
	   ttcp.aio.x -r [-options]     
    Primary options:
	    -r      data sink (server)
	    -t      data source (client)
    Common options:
	    -d      set SO_DEBUG socket option
	    -u      urgent data notification
	    -l ##   length of network read/write buffers (default 8192)
	    -p ##   port number to send to or listen at (default 5001)
	    -A ##   align the start of buffers to this modulus (default 16384)
	    -O ##   start buffers at this offset from the modulus (default 0)
	    -b ##   set SO_SNDBUF and SO_RCVBUF socket buffer size
	    -a ##   number of concurrent outstanding AIO's (default 1)
	    -x ##   number of different buffers to use. (defaults to -a value)
	    -z ##   zcopy threshold.
	    -f X    format for rate: k,K = kilo{bit,byte}
				     m,M = mega{bit,byte}
				     g,G = giga{bit,byte}
	    -v      -t: generate a verifiable pattern.
		    -r: verify the data stream being received.
    Options specific to data source: (-t)
	    -n ##   number of source bufs written to network (default 2048)
	    -N ##   number of seconds to write to network (overrides -n)
	    -D      set TCP_NODELAY socket option (don't buffer TCP writes)
    Options specific to data sink: (-r)
	    -R      set SO_REUSEADDR socket option
	    -L ##   set RCVLOWAT (recv low water mark)

    
  The most common error generated by ttcp.aio.x, resulting in early
  termination is ENOMEM. (-12), The most common cause for this in
  an unloaded system is that SDP failed to mlock the buffers posted
  to the AIO queue, and the most common reason for the lock to fail
  is the lack of lock resource for the user. You should attempt to
  increase the amount of memory that you are allowed to lock, or
  talk to your system administrator. For example the following 
  command in each shell from which you are running ttcp:

    limit memorylocked unlimited
